Static Hand Gesture Recognition Using Capacitive Sensing and Machine Learning.

Automated hand gesture recognition is a key enabler of Human-to-Machine Interfaces (HMIs) and smart living. This paper reports the development and testing of a static hand gesture recognition system using capacitive sensing. Our system consists of a 6×18 array of capacitive sensors that captured five gestures-Palm, Fist, Middle, OK, and Index-of five participants to create a dataset of gesture images. The dataset was used to train Decision Tree, Naïve Bayes, Multi-Layer Perceptron (MLP) neural network, and Convolutional Neural Network (CNN) classifiers. Each classifier was trained five times; each time, the classifier was trained using four different participants' gestures and tested with one different participant's gestures. The MLP classifier performed the best, achieving an average accuracy of 96.87% and an average F1 score of 92.16%. This demonstrates that the proposed system can accurately recognize hand gestures and that capacitive sensing is a viable method for implementing a non-contact, static hand gesture recognition system.

UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals.

With the global spread of the novel coronavirus, avoiding human-to-human contact has become an effective way to cut off the spread of the virus. Therefore, contactless gesture recognition becomes an effective means to reduce the risk of contact infection in outbreak prevention and control. However, the recognition of everyday behavioral sign language of a certain population of deaf people presents a challenge to sensing technology. Ubiquitous acoustics offer new ideas on how to perceive everyday behavior. The advantages of a low sampling rate, slow propagation speed, and easy access to the equipment have led to the widespread use of acoustic signal-based gesture recognition sensing technology. Therefore, this paper proposed a contactless gesture and sign language behavior sensing method based on ultrasonic signals-UltrasonicGS. The method used Generative Adversarial Network (GAN)-based data augmentation techniques to expand the dataset without human intervention and improve the performance of the behavior recognition model. In addition, to solve the problem of inconsistent length and difficult alignment of input and output sequences of continuous gestures and sign language gestures, we added the Connectionist Temporal Classification (CTC) algorithm after the CRNN network. Additionally, the architecture can achieve better recognition of sign language behaviors of certain people, filling the gap of acoustic-based perception of Chinese sign language. We have conducted extensive experiments and evaluations of UltrasonicGS in a variety of real scenarios. The experimental results showed that UltrasonicGS achieved a combined recognition rate of 98.8% for 15 single gestures and an average correct recognition rate of 92.4% and 86.3% for six sets of continuous gestures and sign language gestures, respectively. As a result, our proposed method provided a low-cost and highly robust solution for avoiding human-to-human contact.

Anatomically Designed Triboelectric Wristbands with Adaptive Accelerated Learning for Human-Machine Interfaces.

Recent advances in flexible wearable devices have boosted the remarkable development of devices for human-machine interfaces, which are of great value to emerging cybernetics, robotics, and Metaverse systems. However, the effectiveness of existing approaches is limited by the quality of sensor data and classification models with high computational costs. Here, a novel gesture recognition system with triboelectric smart wristbands and an adaptive accelerated learning (AAL) model is proposed. The sensor array is well deployed according to the wrist anatomy and retrieves hand motions from a distance, exhibiting highly sensitive and high-quality sensing capabilities beyond existing methods. Importantly, the anatomical design leads to the close correspondence between the actions of dominant muscle/tendon groups and gestures, and the resulting distinctive features in sensor signals are very valuable for differentiating gestures with data from 7 sensors. The AAL model realizes a 97.56% identification accuracy in training 21 classes with only one-third operands of the original neural network. The applications of the system are further exploited in real-time somatosensory teleoperations with a low latency of <1 s, revealing a new possibility for endowing cyber-human interactions with disruptive innovation and immersive experience.

Compliance with Barrier Gestures during COVID-19 Pandemic as a Function of the Context: A Longitudinal Observational Survey at the University of Liège.

During the COVID-19 pandemic, barrier gestures such as mask wearing, physical distancing, greetings without contact, one-way circulation flow, and hand sanitization were major strategies to prevent the spread of SARS-CoV-2, but they were only useful if consistently applied. This survey was a follow-up of the first survey performed in 2020 at the University of Liège. We aim to evaluate the compliance with these gestures on campuses and examine differences in the extent of the compliance observed in different educational activities and contexts. During 3.5 months, the counting of compliant and non-compliant behaviors was performed each week in randomly selected rooms. Using data collected during both surveys (2020 and 2021), binomial negative regression models of compliance depending on periods (teaching periods and exam sessions), type of rooms, and campuses were conducted to evaluate prevalence ratios of compliance. The percentage of compliance in this second survey was the highest for mask wearing and physical distancing during educational activities (90% and 88%, respectively) and lowest for physical distancing outside educational activities and hand sanitization (45% and 52%, respectively). Multivariate analyses revealed that the compliance with most gestures was significantly higher in teaching rooms than in hallways and restaurants and during exam sessions. The compliance with physical distancing was significantly higher (from 66%) in auditoriums, where students had to remain seated, than during practical works that allowed or required free movement. Therefore, the compliance with barrier gestures was associated with contextual settings, which should be considered when communicating and managing barrier gestures. Further studies should specify and confirm the determining contextual characteristics regarding the compliance with barrier gestures in times of pandemic.

Video meeting signals: Experimental evidence for a technique to improve the experience of video conferencing.

We found evidence from two experiments that a simple set of gestural techniques can improve the experience of online meetings. Video conferencing technology has practical benefits, but psychological costs. It has allowed industry, education and social interactions to continue in some form during the covid-19 lockdowns. But it has left many users feeling fatigued and socially isolated, perhaps because the limitations of video conferencing disrupt users' ability to coordinate interactions and foster social affiliation. Video Meeting Signals (VMS™) is a simple technique that uses gestures to overcome some of these limitations. First, we carried out a randomised controlled trial with over 100 students, in which half underwent a short training session in VMS. All participants rated their subjective experience of two weekly seminars, and transcripts were objectively coded for the valence of language used. Compared to controls, students with VMS training rated their personal experience, their feelings toward their seminar group, and their perceived learning outcomes as significantly higher. Also, they were more likely to use positive language and less likely to use negative language. A second, larger experiment replicated the first, and added a condition where groups were given a version of the VMS training but taught to use emoji response buttons rather than gestures to signal the same information. The emoji-trained groups did not experience the same improvement as the VMS groups. By exploiting the specific benefits of gestural communication, VMS has great potential to overcome the psychological problems of group video meetings.

SrTiO3 /CuNi-Heterostructure-Based Thermopile for Sensitive Human Radiation Detection and Noncontact Human-Machine Interaction.

Noncontact interactive technology provides an intelligent solution to mitigate public health risks from cross-infection in the era of COVID-19. The utilization of human radiation as a stimulus source is conducive to the implementation of low-power, robust noncontact human-machine interaction. However, the low radiation intensity emitted by humans puts forward a high demand for photodetection performance. Here, a SrTiO3-x /CuNi-heterostructure-based thermopile is constructed, which features the combination of high thermoelectric performance and near-unity long-wave infrared absorption, to realize the self-powered detection of human radiation. The response level of this thermopile to human radiation is orders of magnitude higher than those of low-dimensional-materials-based photothermoelectric detectors and even commercial thermopiles. Furthermore, a touchless input device based on the thermopile array is developed, which can recognize hand gestures, numbers, and letters in real-time. This work offers a reliable strategy to integrate the spontaneous human radiation into noncontact human-machine interaction systems.

Implementing a Hand Gesture Recognition System Based on Range-Doppler Map.

There have been several studies of hand gesture recognition for human-machine interfaces. In the early work, most solutions were vision-based and usually had privacy problems that make them unusable in some scenarios. To address the privacy issues, more and more research on non-vision-based hand gesture recognition techniques has been proposed. This paper proposes a dynamic hand gesture system based on 60 GHz FMCW radar that can be used for contactless device control. In this paper, we receive the radar signals of hand gestures and transform them into human-understandable domains such as range, velocity, and angle. With these signatures, we can customize our system to different scenarios. We proposed an end-to-end training deep learning model (neural network and long short-term memory), that extracts the transformed radar signals into features and classifies the extracted features into hand gesture labels. In our training data collecting effort, a camera is used only to support labeling hand gesture data. The accuracy of our model can reach 98%.

Mysteries of Monozygosity: Theories and Breakthroughs/Twin Research: Rare Case of Lost Twins; Developing a National Twin Registry; Twins' Language and Gesture Delays; DNA Testing for Vanishing Twins/Media Reports: Identical Twins Discordant for COVID Vaccination; World's Oldest Identical Twins; Olympic Athlete Stand-in; Fraternal Twin Football Players.

The processes that give rise to monozygotic (MZ) twins remain elusive. This article describes various theories of MZ twinning that have been examined over the years, although they continue to be speculative. It has also been impossible to know if a singleton began life as an MZ twin; however, a critical technological breakthrough can now reveal this important birth history information with a high degree of certainty. The section that follows presents reviews of current research regarding rare twin loss, development of a twin registry, twins' communicative delays and DNA testing for vanishing twins. The article concludes with a survey of newsworthy twins, namely identical twins discordant for COVID vaccination, the world's oldest identical twins, an Olympic athletic stand-in and fraternal twin football players.

Design and Implementation of a Gesture-Aided E-Learning Platform.

In gesture-aided learning (GAL), learners perform specific body gestures while rehearsing the associated learning content. Although this form of embodiment has been shown to benefit learning outcomes, it has not yet been incorporated into e-learning. This work presents a generic system design for an online GAL platform. It is comprised of five modules for planning, administering, and monitoring remote GAL lessons. To validate the proposed design, a reference implementation for word learning was demonstrated in a field test. 19 participants independently took a predefined online GAL lesson and rated their experience on the System Usability Scale and a supplemental questionnaire. To monitor the correct gesture execution, the reference implementation recorded the participants' webcam feeds and uploaded them to the instructor for review. The results from the field test show that the reference implementation is capable of delivering an e-learning experience with GAL elements. Designers of e-learning platforms may use the proposed design to include GAL in their applications. Beyond its original purpose in education, the platform is also useful to collect and annotate gesture data.

Deep Learning-Based Real-Time AI Virtual Mouse System Using Computer Vision to Avoid COVID-19 Spread.

The mouse is one of the wonderful inventions of Human-Computer Interaction (HCI) technology. Currently, wireless mouse or a Bluetooth mouse still uses devices and is not free of devices completely since it uses a battery for power and a dongle to connect it to the PC. In the proposed AI virtual mouse system, this limitation can be overcome by employing webcam or a built-in camera for capturing of hand gestures and hand tip detection using computer vision. The algorithm used in the system makes use of the machine learning algorithm. Based on the hand gestures, the computer can be controlled virtually and can perform left click, right click, scrolling functions, and computer cursor function without the use of the physical mouse. The algorithm is based on deep learning for detecting the hands. Hence, the proposed system will avoid COVID-19 spread by eliminating the human intervention and dependency of devices to control the computer.

The First Random Observational Survey of Barrier Gestures against COVID-19.

In the context of COVID-19 in Belgium, face-to-face teaching activities were allowed in Belgian universities at the beginning of the 2020-2021 academic year. Nevertheless, several control measures were established to control COVID-19 transmission on the campuses. To ensure compliance with these measures, a random observational survey, based on five barrier gestures, was implemented at the University of Liege (greetings without contact, hand sanitisation, following a one-way traffic flow, wearing a mask and physical distancing). Each barrier gesture was weighted, based on experts' elicitation, and a scoring system was developed. The results were presented as a diagram (to identify the margin of improvement for each barrier gesture) and a risk management barometer. In total, 526 h of observations were performed. The study revealed that some possible improvements could be made in the management of facilities, in terms of room allocation, the functionality of hydro-alcoholic gel dispensers, floor markings and one-way traffic flow. Compliance with the barrier gestures reached an overall weighted score of 68.2 (between 0 and 100). Three barrier gestures presented a lower implementation rate and should be addressed: the use of hydro-alcoholic gel (particularly when exiting buildings), compliance with the traffic flow and the maintenance of a 1.5 m physical distance outside of the auditoriums. The methodology and tool developed in the present study can easily be applied to other settings. They were proven to be useful in managing COVID-19, as the barometer that was developed and the outcomes of this survey enabled an improved risk assessment on campuses, and identified the critical points to be addressed in any further public health communication or education messages.

Video remote interpreting in clinical communication: A multimodal analysis.

OBJECTIVE: Investigating how the spatial and audiovisual conditions in video remote interpreting (VRI) shape communicative interaction in a language-discordant clinical consultation. METHODS: We conducted a multimodal analysis of an authentic VRI-mediated consultation with special reference to spatial arrangements, audiovisual conditions, and the healthcare professional's use of embodied communicative resources (body orientation, eye gaze, gestures). RESULTS: The physician is found to pursue his communicative goals for the consultation by first creating an appropriate spatial and technical environment and then supporting his information-giving and relationship-building actions through the use of nonverbal (embodied) resources like body orientation, gaze and gestures as well as specific turn-management behaviour. CONCLUSION: VRI allows healthcare professionals to access professional interpreters for language-discordant consultations but requires appropriate technical and spatial arrangements as well as users capable of adapting their communicative behaviour to spatial and audiovisual constraints. PRACTICE IMPLICATIONS: Alongside telephone interpreting, VRI is the solution of choice for language-discordant clinical encounters in times of the Covid-19 pandemic. Its use requires appropriate technical and spatial arrangements as well as specific skills on the part of healthcare professionals to cope with inherent audiovisual constraints.